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FOREWORD 


The  ENLISTED  MANPOWER  Work  Unit  conducts  a  continuing  research  program  to 
maintain  and  improve  techniques  and  procedures  for  screening  potential  enlisted  man* 
power.  Objectives  are  1)  to  develop  new  forms  of  screening  manures  for  use  by  the 
Army  and  the  other  services  so  as  to  assess  more  (Effectively  the  trainability  and  usa¬ 
bility  of  potential  enlisted  personnel;  2)  to  develop  new  reference  measures  for  use  as 
standards  in  developing  screening  and  classification  tests  for  all  the  services;  and 
3)  to  improve  methods  for  extracting  predictive  information  from  screening  tests. 

As  one  avenue  to  development  of  technical  information  that  can  contribute  to  more, 
effective  input  screening,  the  feasibility  of  programmed  testing  is  being  investigated. 
BESRL  has  conducted  seveial  experimental  and  theoretical  studies  of  branching  tests  in 
which  testing  is  individualized  by  having  test  questions  so  programmed  that  an  examinee 
who  answers  a  test  item  correctly  is  presented  next  with  a  more  difficult  item  and  an 
examinee  who  answers  incorrectly  is  presented  with  an  easier  item.  By  contrast,  in 
conventional  tests  all  examinees  answer  the  same  items  presented  in  the  same  order. 
The  present  publication  reports  on  a  comparison  of  a  variety  of  computer-simulated 
conventional  and  branching  tests. 

The  entire  ENLISTED  MANPOWER  Work  Unit  is  responsive  to  special  requirements  of 
the  Deputy  Chief  of  Staff  for  Personnel,  as  well  as  to  requirements  of  RDT&E  Project 
2Q024701A721,  "Selection  and  Behavioral  Evaluation,"  FY  1970  Work  Program. 


COMPARISON  OF  COMPUTER-SIMULATED  CONVENTIONAL  AND  BRANCHING  TESTS 


BRIEF 


Requirement: 

To  compare  a  variety  of  ccmputer-simulated  conventional  and  branching  tests  and 
to  extend  the  theoretical  analysis  of  branching  techniques. 


Procedure: 

Computer-simulated  tests  of  two  types  were  compared.  One  type  consisted  of  con¬ 
ventional  tests  varying  in  length  (5,  10,  1b  items)  and  distribution  of  item  difficulty 
indexes  (all  items  at  p  t  .50;  normal,  p  x  .30  -  .70  and  p  >  .10  -  .90;  and  rectilinear, 
p  s  .30  -  .70  and  p  ■  .10  -  .90) .  The  second  consisted  of  branching  tests  varying  in  length 
(5,  10,  15  items  to  be  answered  by  each  examinee) ,  number  of  items  presented  at  each 
level  of  difficulty  (1,2) ,  and  distribution  of  item  difficulties  comparable  to  those  of  the 
conventional  tests.  In  addition,  both  types  of  test  were  varied  in  assumed  item  validity. 
The  comparisons  were  made  in  terms  of  correlation  between  test  scores  and  underlying 
ability  (Lord's  model) . 


Findings. 

In  tests  with  higher  item  validities  (rbjs  ,  .60  -  .90) ,  a  branching  test  had  a  higher 
correlation  with  underlying  ability  than  did  any  conventional  test,  for  all  three  lengths 
studied. 


Applicability: 

This  theoretical  analysis  supports  an  exploratory  experimental  study  previously 
made.  It  indicates  the  research  promise  of  tests  with  branching  programs  and  provides 
useful  guidelines  for  the  design  of  further  studies  of  progiammed  tests. 
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COMPARISON  OF  COMPUTER-SIMULATED  CONVENTIONAL  AND  BRANCHING  TESTS 


In  line  with  its  interest  in  unconventional  testing  techniques,  the 
Behavior  and  Systems  Research  Laboratory  (BESRL)  has  conducted 
several  research  studies  of  the  branching  techniques.  Waters  (1),  in  a 
theoretical  study,  found  that  a  hypothetical  five-item  branching  test 
correlated  somewhat  higher  with  underlying  ability  than  did  hypothetical 
five-item  conventional  tests.  Bayroff  and  Seeley  (2)  obtained  empirical 
Indications  that  correlation  between  short  branching  tests  and  long  con¬ 
ventional  tests  was  considerably  higher  than  the  expected  correlation 
between  equally  short  conventional  tests  and  long  conventional  tests. 

The  present  report  continues  the  theoretical  analysis  and  compares  a 
variety  of  computer- simulated  conventional  and  branching  tests. 

In  the  usual  testing  situation,  each  examinee  takes  all  the  items, 
and  item  sequence  is  the  same  for  each  examinee.  It  is  possible,  how¬ 
ever,  to  have  sequential  or  branching  tests  in  which  all  examinees  do 
not  take  the  same  items  and  the  sequence  of  item  presentation  for  an 
individual  is  some  function  of  his  performance  on  previous  items;  that 
is,  an  item  answered  correctly  is  followed  by  a  more  difficult  item,  an 
item  answer  incorrectly,  by  a  less  difficult  item.  The  rationale  for 
the  latter  procedure  is  that  presentation  of  items  based  on  an  examinee's 
past  performance  allows  each  individual  to  take  items  that  are  progres¬ 
sively  more  appropriate  to  his  own  level  of  ability.  It  is  conceivable 
that  such  a  procedure  would  reduce  testing  time,  and  for  a  given  amount 
of  time  would  permit  more  accurate  measurement  of  an  individual's  ability, 
principally  by  reducing  opportunities  for  chance  success  by  low  ability 
examinees'  attempting  items  too  difficult  for  them. 


TESTS 


Conventional  Tests 

Five-,  ten-,  and  fifteen-item  hypothetical  conventional  (c)  tests 
were  evaluated.  All  tests  were  symmetric  around  p  ■  .50,  but  varied  in 
item  difficulty  distributions.  The  distributions  investigated  were  all 
items  at  p  **  .50  (C50),  roughly  normal  (CN),  or  rectilinear  ( CR) .  Each 
of  the  CN  and  CR  tests  was  tried  out  with  difficulty  ranges  of  .30 
through  .70  and  .10  through  .90.  Table  1  gives  the  C50,  CN,  and  CR 
item  difficulty  distributions  for  the  five-,  ten-,  and  fifteen-item 
conventional  tests. 


for  conventional  tests 
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Branching  Tests 


One- Item- Per-Stage  Tests.  Six  hypothetical  Qne-item-per-stage  (1-PS) 
branching  tests  were  evaluated.  The  structure  of  these  six  tests  is  shown 
in  Figures  1  through  6,  together  with  the  difficulty  of  each  item.  Two 
tests  were  studied  at  each  of  the  three  tests  lengths  (5,  10,  and  15  items). 
One  of  the  two  tests  covered  a  difficulty  range  of  .50  through  .70  and 
the  other  ranged  from  .10  through  .90.  The  five- item-per- subject  branch¬ 
ing  tests  contained  15  items  with  each  examinee  responding  to  only  five 
of  the  items.  In  the  ten-item-per-subject  tests,  each  examinee  took  ten 
of  the  55  items  in  the  test.  The  fifteen-item-per-subject  tests  were 
composed  of  120  items.  In  each  of  the  six  tests,  the  first  item  (p  ■  .50) 
was  the  same  for  all  examinees,  but  the  remaining  items  taken  were  deter¬ 
mined  by  the  examinee's  performance  on  the  immediately  preceding  item. 

If  an  examinee  passed  an  item,  he  proceeded  to  a  more  difficult  one;  if 
he  failed  an  item,  he  proceeded  to  an  easier  one.  When  the  range  of 
p- values  in  a  test  was  .30  through  .70,  increases  and  decreases  in  diffi¬ 
culty  between  adjacent  items  were  in  steps  of  .05  for  the  f  ive-item-per- 
subject  test,  .0222  for  the  ten-item-per-subject  test,  and  .0145  for  the 
fifteen-item-per-subject  test.  For  the  .10  through  .Q0  range  tests,  the 
steps  were  .10  for  the  five- item- per-subject  test,  .0444  for  the  ten-item- 
per-subject  test,  and  .0286  for  the  fifteen-item-per-subject  test. 

Two- Item- Per-Stage  Tests.  Four  hypothetical  two-item-per-stage 
(2-PS),  ten-item-per-subject  branching  tests  were  evaluated.  The  struc¬ 
ture  of  these  four  tests,  and  the  distributions  of  item  difficulties,  is 
shown  in  Figure  7*  Each  of  these  tests  was  composed  of  114  items.  At 
each  stage  in  these  tests,  the  examinees  took  two  items  of  the  same  dif¬ 
ficulty  level .  The  first  two  items  taken  by  all  examinees  had  p-values 
of  .50.  If  the  examinee  passed  both  items  in  a  pair,  he  branched  to  a 
more  difficult  item  pair;  if  he  passed  one  of  the  items  in  a  pair,  he 
branched  to  a  pair  of  equal  difficulty;  if  he  failed  both  items  in  a 
pair,  he  proceeded  to  an  easier  pair  of  items.  Items  for  two  of 
the  tests  covered  a  difficulty  range  of  .50  through  .70,  while  the  other 
two  tests  ranged  from  .10  through  .00.  For  each  of  these  difficulty 
ranges,  one  branching  tests  was  developed  by  having  equally  spaced  item 
pairs  in  the  terminal  row  of  the  test  (2-PS-E).  The  p-values  of  the  item 
pairs  in  the  other  rows  were  determined  from  the  terminal  item  pair  values. 
For  the  other  two-item-per-stage  tests,  2-PS-U  (one  for  each  of  the  item 
difficulty  ranges),  the  item  pair  p-values  were  determined  by  branching 
downward  from  the  p  =  .50  item  pair  to  the  terminal  row  of  item  pairs. 
Using  this  procedure,  the  item  pairs  in  the  terminal  rows  were  not 
equally  spaced  as  in  the  2-PS-E  tests  but  were  spaced  so  that  the  in¬ 
tervals  between  item  pairs  were  smaller  in  the  middle  part  of  the  diffi¬ 
culty  ranges,  and  larger  nearer  the  extreme  difficulty  values.  Scores 
for  all  four  of  the  two-item-per-stage  tests  ranged  from  0  to  62. 
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Figure  1 .  One^em-per-stage,  five-item  branching  test  with  a  difficulty  range 
of  .30-.70 
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Figure  2.  One-item-per-stage,  five-item  branching  test  with  a  difficulty 
of  .10-.90 
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Figure  3.  One-item-per-stage,  ten-item  branching  test  with  a  difficulty  range  of  .30-. 70 
! Difficulty  values  for  items  shown  are  given  on  page  7> 
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DIFFICULTY  VALUES  FOR  ONE- ITEM- PER- STAGE,  TEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .30-. 70  (Figure  3) 
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Figure  4.  One-item-per-stage,  ten-item  branching  test  with  a  difficulty  range  of  .10-.90 
f Difficulty  values  for  items  shown  are  given  on  page  91 
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DIFFICULTY  VALUES  FOR  ONE- ITEM- PER- STAGE,  TEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .10-. 90  (Figure  4) 


Item  No. 

Dlff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 
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.5000 

6 

.4112 

24 

.5888 

42 

.4112 

7 

.6332 

25 

.5000 

43 

.3224 

8 

.5444 

26 

.4112 

4  } 

.2336 

9 

.4556 

27 

.3224 

45 

.1448 

10 

.3668 

28 

•2336 

46 

.8996 

11 

.6776 

29 

.8108 

47 

.8108 

12 

.5888 

30 

.7220 

48 

.7220 

13 

.5000 

31 

.6532 

49 

.6332 

14 

.4112 

32 
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50 
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51 
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53 

.2780 

18 
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36 
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Figure  5.  One-item-per-stage,  fifteen-item  branching  test  with  a  difficulty  range  of  .30-. 70 

(Difficulty  values  for  items  shown  are  given  on  pages  It  and  121 
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DIFFICULTY  VALUES  FOR  ONE- ITEM- PER- STAGE,  FIFTEEN- ITEM  BRANCHING  TEST 

WITH  DIFFICULTY  RANGE  OF  .30- .70  (Figure  5) 
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Figure  6.  One-item-per-stage,  fifteen-item  branching  test  with  a  difficulty  range  of  .10-.90 
(Difficulty  values  for  items  shown  are  given  on  pages  14  and  151 
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DIFFICULTY  VALUES  FOR  ONE- ITEM- PER- STAGE,  FIFTEEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .10- .QO  (Figure  6) 


Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

1 

.5000 

21 

•3572 

41 

.5000 

2 

•  5285 

22 

.6713 

42 

.4429 

3 

.4714 

23 

.6142 

43 

.3858 

4 

•  5571 

24 

•5571 

44 

.3287 

5 

.5000 

25 

.5000 

45 

.2716 

6 

.4429 

26 

.4429 

46 

•7569 

7 

.5856 

27 

.3858 

47 

.6998 

8 

.528-5 

28 

.3287 

48 

.6427 

9 

.4714 

29 

.6998 

49 

.5856 

10 

.4143 

30 

.6427 

50 

.5285 

11 

.61 42 

31 

•  5856 

51 

.4714 

12 

.557I 

32 

.5285 

52 

.4143 

13 

.6000 

33 

.4714 

53 

.3572 

14 

.4429 

34 

.4143 

54 

.3001 

15 

.3858 

35 

.3572 

55 

.2430 

16 

.6427 

36 

.3001 

56 

.7855 

17 

.5856 

37 

.7284 

57 

.7284 

18 

.6285 

38 

.6713 

58 

.6713 

19 

.4714 

39 

.6142 

59 

.6142 

20 

.4143 

40 

•5571 

60 

•5571 

DIFFICULTY  VALUES  FOR  ONE- ITEM- PER- STAGE,  FIFTEEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .10- -90  (Continued) 


Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

61 

•  5000 

81 

.7284 

101 

•3572 

62 

.4429 

82 

.6713 

102 

.3001 

63 

.3858 

83 

.6142 

103 

.2430 

64 

.3287 

84 

•5571 

104 

.1859 

65 

.2716 

85 

•  5000 

105 

.1288 

66 

.2145 

86 

.4429 

106 

.8997 

67 

.8140 

87 

.3858 

107 

.8426 

68 

•7569 

88 

.3287 

108 

.7855 

6Q 

.6998 

89 

.2716 

109 

.7284 

70 

.6427 

90 

.2145 

110 

•6713 

71 

.58% 

91 

.1574 

111 

.6142 

72 

.5285 

92 

.8711 

112 

•5571 

73 

.4714 

93 

.8140 

113 

•  5000 

74 

.4143 

94 

.7569 

114 

.4429 

75 

•3572 

95 

.6998 

115 

.3858 

76 

.3001 

96 

.6427 

116 

.3287 

77 

.2430 

97 

.5856 

117 

.2716 

78 

.1859 

98 

.5285 

118 

.2145 

79 

.8426 

99 

.4714 

119 

•  1574 

8o 

.7855 

100 

.4143 

120 

.1003 

DIFFICULTY  VALUES  FOR  TWO- ITEM- PER- STAGE- E,  TEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .30-.7O  ('Figure  7) 


Item  No . 

Diff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

1 

.5000 

20 

•  3933 

39 

•  4733 

2 

.5000 

21 

•  3400 

40 

•  4733 

3 

.6067 

22 

.3400 

41 

.4467 

4 

.6067 

23 

.6867 

42 

.4467 

5 

•  5000 

24 

.6867 

43 

.4200 

6 

.5000 

23 

.6600 

44 

.4200 

7 

.3933 

26 

.6600 

45 

•  3933 

8 

•  >933 

27 

•  6333 

46 

•  3933 

9 

.6600 

28 

•  6333 

47 

.3667 

10 

.660*9 

29 

.6067 

48 

.3667 

11 

.6067 

30 

.606” 

49 

•  3400 

12 

.6067 

31 

.5800 

00 

•  3400 

13 

•  3833 

32 

.3800 

51 

•  3133 

14 

•  3333 

33 

•3333 

52 

.3133 

15 

.5000 

34 

•  3333 

53 

•  7000 

16 

.5000 

35 

.5267 

54 

.7000 

17 

.4^67 

36 

•  5267 

55 

.6867 

18 

.4467 

37 

.3000 

56 

.6867 

19 

•  3933 

38 

.5000 

57 

•  6733 
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DIFFICULTY  VALUES  FOR  TWO- ITEM- PER- STAGE- E .  TEN- ITEM  BRANCHING  TEST 


WITH  DIFFICULTY  RANGE 

Item  No. 

Diff.  Value 

Item  No. 

58 

.6733 

77 

5Q 

.6600 

78 

60 

.6600 

79 

61 

.6467 

80 

62 

.6467 

81 

63 

.6333 

82 

64 

.6333 

83 

65 

.6200 

84 

66 

.6200 

85 

67 

.6067 

86 

68 

.6067 

87 

69 

.5933 

88 

70 

•5933 

89 

71 

.58OO 

90 

72 

.5800 

91 

73 

.5667 

92 

74 

.5667 

93 

75 

•5533 

94 

76 

•5533 

95 

OF  .30- .70(  Continued) 


Diff.  Value 

Item  No . 

Diff.  Value 

•  5400 

96 

.4200 

.5400 

97 

.4067f 

.5267 

98 

.4067 

.5267 

99 

•3933 

.5133 

100 

.3933 

.5133 

101 

.3800 

.5000 

102 

.3800 

.5000 

103 

•  3667 

.4867 

104 

.3667 

.4867 

105 

•  3533 

.4733 

106 

•3533 

.4733 

107 

•  340C 

.4600 

108 

.3400 

.4600 

109 

.3267 

.4467 

110 

.3267 

.4467 

111 

.3133 

•  4333 

112 

.3133 

•  4333 

113 

.3000 

.4200 

114 

.3000 
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DIFFICULTY  VALUES  FOR  TWO-ITEM- 
WITH  DIFFICULTY  RANGE 

PER-STAGE-U,  TEN- ITEM  BRANCHING  TEST 

OF  .30-.' 70  (Figure  7) 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

1 

.‘5000 

20 

.4500 

39 

.4875 

2 

.5000 

21 

.4000 

40 

.4875 

3 

•5500 

22 

.4000 

41 

•4750 

4 

.5500 

23 

.6500 

42 

•4750 

5 

.5000 

24 

.6500 

43 

.4625 

6 

.5000 

25 

.6000 

44 

.4625 

7 

.4500 

26 

.6000 

45 

.4500 

8 

.4500 

27 

.5750 

46 

.4500 

9 

.6000 

28 

■  5750 

47 

.4250 

10 

.6030 

20 

.5500 

48 

.4250 

11 

•  5530 

30 

.5500 

49 

.4000 

12 

•  55^0 

31 

•5375 

50 

.4000 

13 

.‘>250 

32 

•  5375 

51 

•  3500 

14 

•  5850 

33 

.5250 

52 

•  3500 

15 

.5000 

34 

.5250 

53 

.7000 

16 

•  5300 

35 

.5125 

54 

.7000 

I  17 

.4750 

36 

•5125 

55 

.6500 

18 

.4750  ' 

37 

.5000 

56 

.6500 

19 

.4500 

38 

.5000 

57 

.6250 

DIFFICULTY  VALUES  FOR  TWO- ITEM- PER- STAGE-U,  TEN- ITEM  BRANCHING  TEST 
WITH  DIFFICULTY  RANGE  OF  .30-.70  (Continued) 


Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value 

58 

.6250 

77 

.5187 

96 

.4625 

5Q 

.6000 

78 

.5187 

97 

.456? 

60 

.6000 

79 

.5125 

98 

.4562 

61 

•5875 

80 

.5125 

99 

.4500 

62 

.5875 

81 

.5062 

100 

.4500 

65 

•5750 

82 

•  5062 

101 

•  4575 

64 

•5750 

85 

.5000 

102 

.4575 

65 

.5625 

84 

.5000 

105 

.4250 

66 

.5625 

85 

•4957 

104 

.4250 

67 

•  5500 

86 

.4957 

105 

.4125 

68 

.5500 

87 

.4875 

106 

.4125 

69 

.5457 

88 

.4875 

107 

.4000 

70 

-5457 

89 

.4812 

108 

.4000 

71 

•5575 

90 

.4812 

109 

•5750 

72 

•  5575 

91 

.4750 

110 

.5750 

75 

-5512 

92 

.4750 

111 

•  5500 

74 

.5512 

95 

.4687 

112 

.5500 

75 

.5250 

94 

.4687 

115 

.5000 

76 

.5250 

95 

.4625 

114 

.5000 

-  20  - 


DIFFICULTY  VALUES  FOR  TWO-ITEM- 
WITH  DIFFICULTY  RANGE 

PER- STAGE- E, 
OF  .10-. 90 

TEN- ITEM  BRANCHING  TEST 
(Figure  7) 

Item  No. 

Diff.  Value 

Item  No. 

Dlff.  Value  .Item  No. 

Diff.  Value 

1 

•  5000 

20 

.2867 

39 

.4467 

2 

•  5000 

21 

.1800 

40 

.4467 

3 

.7133 

22 

.1800 

41 

•3933 

4 

.7133 

23 

.8733 

42 

•  3933 

3 

•  5C00 

24 

.8733 

43 

.3400 

6 

.5000 

25 

.8200 

44 

.3400 

7 

.2867 

26 

.8200 

45 

.2867 

8 

.2867 

27 

.7667 

46 

.2867 

9 

.8200 

28 

.7667 

47 

.2333 

10 

.8200 

29 

•7133 

48 

.2333 

11 

.7133 

30 

•7133 

49 

.1800 

12 

.7133 

31 

.6600 

50 

.1800 

13 

.6067 

32 

.6600 

51 

.1267 

14 

.6067 

33 

.6067 

52 

.1267 

15 

.5000 

34 

.6067 

53 

.9000 

16 

.500" 

35 

•  5533 

54 

.9000 

17 

•  3933 

36 

•  5533 

55 

.8733 

18 

•  3953 

37 

.5000 

56 

.8733 

19 

.2867 

38 

.5000 

57 

.8467 
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DIFFICULTY  VALUES  FOR  TWO- ITEM - 
WITH  DIFFICULTY  RANGE 

Item  No.  Diff.  Value  Item  No. 

PER-  STAGE- E,  TEN- ITEM  BRANCHING  TEST 

OF  .10-. °0  (Continued) 

Diff.  Value  Item  No.  Diff.  Value 

58 

.8467 

77 

.5800 

96 

.3400 

59 

.8200 

78 

.5800 

97 

.3133 

60 

.8200 

79 

•5533 

98 

•3133 

61 

•7933 

80 

•5533 

99 

.2867 

62 

•7953 

81 

.5267 

100 

.2867 

65 

.7667 

82 

.5267 

101 

.2600 

64 

.7667 

83 

.5000 

102 

.2600 

65 

•7400 

84 

.5000 

103 

.2333 

66 

.7400 

85 

.4733 

104 

.2333 

67 

.7133 

86 

.4733 

105 

.2067 

68 

.7133 

87 

.4467 

106 

.2067 

69 

.6867 

88 

.4467 

107 

.1800 

70 

.6867 

85 

.4200 

108 

.1800 

71 

.6600 

90 

.4200 

109 

•  1533 

72 

.6600 

91 

•  3933 

no 

.1533 

73 

•  6333 

92 

.3933 

111 

.1267 

74 

.6333 

93 

.3667 

112 

.1267 

75 

.6067 

94 

.3667 

113 

.1000 

76 

.6067 

95 

.3400 

114 

.1000 
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DIFFICULTY  VALUES  FOR  TWO-ITEM- 
WITH  DIFFICULTY  RANGE 

Item  No.  Diff .  Value  Item  No. 

PER-STAGE-U,  TEN- ITEM  BRANCHING  TEST 

OF  .10- .90  (Figure  7) 

Diff.  Value  Item  No.  Diff.  Value 

1 

.5000 

20 

.4000 

39 

•  4750 

2 

.5000 

21 

•  3000 

40 

.4750 

3 

.6000 

22 

•  3000 

41 

.4500 

4 

.6000 

23 

.8000 

42 

.4500 

5 

.5000 

24 

.8000 

43 

.4250 

6 

•  5000 

25 

.7000 

44 

.4250 

7 

.4000 

26 

.7000 

45 

.4000 

8 

.4000 

27 

.6500 

46 

.4000 

9 

•7000 

28 

.6500 

47 

•  3500 

10 

.7000 

29 

.6000 

48 

•  3500 

11 

.6000 

30 

.6000 

49 

.3000 

12 

.6000 

31 

•5750 

50 

.3000 

!3 

.5500 

32 

•5750 

51 

.2000 

14 

.5500 

33 

•  5500 

52 

.2000 

13 

.5000 

34 

•5500 

53 

.9000 

16 

.5000 

35 

.5250 

54 

.9000 

17 

.4500 

36 

.5250 

55 

.8000 

18 

.4500 

37 

.5000 

56 

.8000 

19 

.4000 

38 

.*1000 

57 

•7500 

DIFFICULTY  VALUES  FOR  TWO-ITEM- 
WITH  DIFFICULTY  RANGE 

PER- 

OF 

STAGE- U, 

.10-. 90 

TEN- ITEM  BRANCHING  TEST 
( Continued) 

Item  No. 

Diff.  Value 

Item  No. 

Diff.  Value  Item  No. 

Diff.  Value 

58 

.7500 

77 

•5575 

96 

.4250 

59 

.7000 

78 

■5575 

97 

.4125 

60 

.7000 

79 

.5250 

98 

.4125 

61 

.6750 

80 

.5250 

99 

.4000 

62 

.6750 

81 

.5125 

lOO 

.4000 

65 

.6500 

82 

.5125 

101 

•5750 

64 

.6500 

85 

.5000 

102 

■5750 

65 

.6250 

84 

.5000 

105 

■5500 

66 

.6250 

85 

.4875 

104 

•  5500 

67 

.6000 

86 

.4875 

105 

•5250 

68 

.6000 

87 

.4750 

106 

.5250 

69 

.5875 

88 

•  4750 

107 

.5000 

70 

.5875 

89 

.4625 

108 

.5000 

71 

•5750 

90 

.4625 

109 

.2500 

72 

•5750 

91 

.4500 

110 

.2500 

75 

.5625 

92 

.4500 

111 

.2000 

74 

.5625 

95 

.4575 

112 

.2000 

75 

•5500 

94 

■4575 

115 

.1000 

76 

.5500 

95 

.4250 

114 

.1000 

COMPUTATIONAL  PROCEDURES  AND  ASSUMPTIONS 

Statistical  computations  were  based  on  a  theoretical  model  presented 
by  Lord  (3).  The  model  assumes  that  there  is  a  trait  or  ability  under¬ 
lying  the  raw  scores  on  a  test,  and  that  the  probability  of  an  examinee's 
responding  correctly  to  a  test  icem  is  a  normal  ogive  function  of  his 
position  on  the  ability  dimension.  Since  item  responses  are  a  function 
only  of  scores  on  the  ability  continuum,  they  are  independent  of  each 
other  when  ability  is  held  constant.  When  all  the  items  in  a  test  are 

2 

assumed  to  have  the  same  biserial  correlation  (R^)  with  ability,  is 

an  estimate  of  item  intercorrelation.  Three  major  steps  are  involved  in 
obtaining  the  correlation  between  test  score  and  underlying  ability: 

The  proportion  of  examinees  passing  each  item  is  determined  for  each  of 
the  abil ity  level s  under  consideration;  the  conditional  distribution  of 
test  scores  is  obtained  for  each  ability  level;  and  the  bivariate  fre¬ 
quency  distribution  of  test  score  and  ability  is  obtained. 


Proportion  of  Examinees  at  a  Given  Level  of  Ability  Who  Pass  an  Item 

When  the  group  tested  is  assumed  to  be  normally  distributed  on 
ability.  Lord's  formulas  (9)  and  (10)  may  be  used  to  find  the  proportion 
of  examinees  who  pass  each  of  the  test  items  when  ability  is  held  con¬ 
stant.  In  Lord's  notation,  a  value  of  g^  (the  z  score  corresponding  to 

the  p-value  of  item  i  at  a  specified  ability  level)  is  computed  for  each 
ability  level  under  consideration  by  formula  (9): 

h  -  Ri  •  c 
*1 - ^ -  •  w^ere 

h^  =  the  z  value  corresponding  to  the  population  p-value  of  item  i 
R^  ■  the  biserial  correlation  between  item  i  and  underlying  ability 
c  =  the  z  score  representing  the  ability  level  being  considered 


Each  g^  is  converted  to  (p-value  of  item  i  for  examinees  at  a  given 
ability  level)  by  Lord's  formula  (10): 

P^  =  A  (g^)  =  area  of  normal  curve  above  the  point  g^ . 

These  values  are  computed  for  each  ability  level. 
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Conditional  Test  Score  Distribution  for  Given  Ability  Levels 

For  conventional  tests,  the  distribution  of  test  scores  at  each  of 
the  specified  ability  levels  may  be  computed  by  expansion  of  Lord's 
formula  (11):  * 

n 

tt  (P.  +  Q  ),  where 
i  -  1  1  1 


n 

tt  indicates  the  successive  multiplication  of  the  ( P .  +0.)  terms 
i  -  1  1 

n  ■  number  of  items  in  test 

Pj  *  proportion  passing  item  i  for  the  given  ability  level 
0  -  1  -  P 

v<i  x 

Terms  of  this  expansion  give  all  possible  ways  of  obtaining  various  test 
scores.  Those  terms  which  lead  to  the  same  test  score  are  sutrsned  to  obtain 
the  distribution  of  test  scores  for  a  given  ability  level. 

Although  Lord  does  not  discuss  branching  tests,  his  model  is  also 
applicable  to  this  type  of  test.  For  a  branching  test,  the  proportion 
of  examinees  (at  a  specif:'*!  ability  level)  following  any  given  path  may 
be  determined  by  multiplying  the  P^  or  values  (as  obtained  by  Lord's 

formulas  9  and  10 )  of  the  items  which  make  up  that  path.  If  an  item  is 
passed,  its  P^  value  is  used;  if  an  item  is  failed,  its  value  is  used. 

Such  a  proportion  is  computed  for  each  path,  and  values  for  paths  leading 
to  the  same  test  score  are  summed  to  obtain  the  test  score  distribution. 


Bivariate  Frequency  Distribution  of  Test  Score  and  Ability 

For  both  conventional  and  branching  tests,  the  bivariate  distribution 
of  test  score  and  ability  is  obtained  by  multiplying  the  conditional  test 
score  distribution  for  each  ability  level  by  the  ordinate  value  of  the 
normal  curve  at  that  ability  level  (Lord's  formula  14,  applicable  when  a 
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normal  distribution  of  ability  is  assumed).  The  test-ability  correlation 
coefficient  may  be  computed1-'  from  this  scatterplot. 

In  the  present  study,  the  distribution  of  underlying  ability  in  the 
theoretical  sample  of  examinees  wf.s  assumed  to  be  normal  with  X  ■  0  and 
a  ■  1.00.  Twenty-nine  levels  of  ability,  measured  in  standard  scores 
ranging  from  +3*5  to  -3-5  in  steps  of  .23  were  used.  The  biserial  corre¬ 
lation  between  an  item  and  ability  was  constant  for  all  items  in  a  given 
test.  For  each  of  the  five-  and  ten-item-per-subject  conventional  and 
branching  tests  evaluated,  the  assumed  biserial  was  varied  from  .30  to 
•90  in  steps  of  .10.  The  fifteen-item  tests  were  evaluated  at  biserials 
of  .40,  .60  and  .80. 


RESULTS  AND  DISCUSSION 

Five-Item  Tests 

Conventional  Tests.  The  correlation  coefficients  between  test  score 
and  ability  for  the  five-item  conventional  tests  are  shown  in  the  first 
three  rows  of  Table  2.  For  biserials  of  .30  through  .70  (rij  “  *09 

through  .49),  the  all  .50  (C50)  test  obtained  the  highest  coefficients, 
and  the  .3 P'  through  .7  CR  test  yielded  a  higher  relationship  to  the 
ability  criterion  than  did  the  .1  through  .9  CR  test.  At  the  .80  biserial 
(r^j  *  -64),  the  .3  through  .7  CR  test  yielded  the  highest  coefficient, 

and  the  C50  test  was  next.  Finally,  at  the  assumed  biserial  of  .90 
(r^j  ■  *81),  the  wide  range  rectilinear  test  (.1  through  .9  CR  test)  had 

the  highest  correlation  coefficient,  and  the  C50  test  had  the  lowest 
coefficient  of  the  three  conventional  tests.  Overall,  the  C30  was  best 
for  low  to  moderate  item  intercorrelation;  the  moderate  range  (  .3  through 
.7)  and  eventually  the  wider  range  (.1  through  .9)  tests  were  best  for 
higher  intercorrelations. 


^  A  FORTRAN  program  which  performs  these  computations  was  written  for  the 
GE  223  computer®'  by  Mr.  Sidney  Sachs  of  the  Computer  Applications  Branch, 
Behavior  and  Systems  Research  Laboratory.  This  program  was  used 
to  obtain  the  test-ability  coefficients  reported.  It  should  be  noted 
that  Brogden  (4),  Tucker  (5),  and  Lord  (3)  have  provided  computationally 
easier  formulas  for  obtaining  the  test-ability  coefficients  for  conven¬ 
tional  tests. 

^  The  commercial  designation  is  used  only  in  the  interest  of  specificity 
in  reporting.  Its  use  does  not  constitute  indorsement  by  the  Army  or 
by  BESRL. 

—  For  simplicity  in  presentation,  difficulty  values  mentioned  in  the  text 
are  hereafter  expressed  in  one  decimal  place  in  contrast  to  the  biserials 
which  are  in  two  decimal  places. 


-  27  - 


T 


% 


Branching  Tests.  The  results  for  the  five-item-per-subject  branch¬ 
ing  tests  are  shown  in  the  last  two  rows  of  Table  2.  The  coefficient  for 
the  moderate  range  .3  through  '  .7  test  was  higher  than  that  for  the  wider 
range  test  for  assumed  biserials  of  .60  through  .80;  the  .1  through  .9 
range  test  had  the  higher  coefficient  at  r^  =  .90. 

Comparison  of  Conventional  and  Branching  Tests.  One  of  the  branch¬ 
ing  tests  was  superior  to  any  of  the  conventional  tests  for  r^  ^  *60 

( r^j  2r  -36).  At  the  higher  biserials,  .70  through  .90,  branching 

tests  yielded  higher  coefficients  than  did  any  of  the  conventional  tests. 
For  the  lower  biserials,  .30  through  .60,  the  C^?0  conventional  tests  re¬ 
sulted  in  slightly  higher  coefficients  than  did  either  of  the  branching 
tests . 


Table  2 


TEST  SCORE-ABILITY  CORRELATION  COEFFICIENTS  FOR  FIVE-ITEM-PER-SUBJECT 

CONVENTIONAL  AND  BRANCHING  TESTS 


Biserials 

•  30 

.40 

•50 

.60 

.70 

.80 

.90 

Test 

Correlation  Coefficients* 

C  (all  .50) 

482 

601 

696 

769 

823 

858 

87I 

C  (.3  -  .7,  R) 

473 

591 

68e 

762 

819 

861 

887 

C  (  .1  -  .9,  R) 

43- 

649 

646 

726 

793 

850 

900 

B  (  .3  -  -7,  1-PS) 

478 

599 

694 

774 

835 

880 

906 

B  (  .1  -  .9,  1-PS) 

461 

580 

680 

760 

826 

878 

920 

a  Decimal  points  omitted. 


Ten-Item  Tests 

Conventional  Tests.  The  test  score-ability  correlation  coefficients 
for  the  ten-item  conventional  tests  are  shown  in  the  first  five  rows  of 
Table  3*  The  C60  test  had  the  highest  coefficient  for  each  biserial 
through  .60.  For  these  same  biserials,  all  the  .3  throigh  .7  range  tests 
were  next  highest  and  the  .1  through  .9  range  tests  were  lowest.  At 
r.  ■  .70,  the  060  and  .30  through  .7  tests  were  about  equally  effective, 

D  X  S 

and  yielded  higher  coefficients  than  the  .1  through  .9  tests.  At  biserials 
.80  and  .90,  the  original  situation  was  reversed  and  the  C50  test  had  the 
lowest  coefficients  and  the  .1  through  .9  tests  the  highest  coefficients. 
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Branching  Tests.  The  ten-item-per-subject  branching  test  data  are 
given  in  the  bottom  six  rows  of  Table  3.  The  .3  through  .7  1-PS  tests 
tended  to  correlate  higher  with  the  criterion  than  did  the  .1  through  .9 
tests  through  a  biserial  of  .60.  Above  this  level  the  converse  held.  Tt 
should  be  noted  that  for  all  biserials,  and  any  given  item  difficulty 
range,  the  1-PS  branching  test  correlated  higher  than  any  2-PS  test 
covering  the  same  range.  In  fact,  with  only  one  exception  (r^s  "  *90 ), 

both  the  1-PS  .1  through  .9  and  .3  through  .7  tests  yielded  higher 
coefficients  than  did  any  of  the  2- PS  tests.  The  2-PS-E  tests  correlated 
higher  with  the  ability  criterion  than  did  the  2-PS-U  tests. 


Table  3 

TEST  SCORE-ABILITY  CORRELATION  COEFFICIENTS  FOR  TEN-ITEM-PER-SUBJECT 

CONVENTIONAL  AND  BRANCHING  TESTS 


Biserials 

•  50 

.40 

•  50 

.60 

.70 

.80 

.90 

Test 

Correlation  Coefficients* 

C  (all 

•5) 

614 

728 

807 

850 

891 

905 

898 

c  (.3  - 

•7, 

N) 

608 

723 

802 

856 

890 

909 

910 

c  (.3  - 

•7, 

R) 

604 

719 

799 

854 

890 

911 

917 

c  (.1  - 

•9, 

N) 

386 

702 

786 

844 

886 

913 

929 

c  (  .1  - 

•9, 

R) 

563 

680 

767 

830 

877 

913 

941 

B  (.3  - 

•7, 

1-PS) 

612 

728 

808 

866 

904 

926 

931 

B  (.3  - 

•7, 

2-PS-E) 

520 

642 

737 

809 

863 

898 

915 

B  (.3  - 

•7, 

2-PS-U) 

512 

633 

721 

799 

851 

885 

898 

B  (  .1  - 

•9, 

1-PS) 

601 

719 

801 

862 

905 

934 

953 

B  (  .1  - 

•9, 

2-PS-E) 

531 

659 

751 

825 

881 

921 

948 

B  (.1  - 

•9, 

2-PS-U) 

519 

640 

729 

808 

862 

899 

918 

1  Decimal  points  omitted. 


Comparison  of  Conventional  and  Branching  Tests.  One  of  the  1-PS 

branching  tests  was  superior  to  any  of  the  conventional  tests  for 

biserials  above  .40  (the  .3  through  .7  1-PS  was  highest  at  r^r  -  .50 

and  .60;  the  .1  through  .9  1-PS  was  highest  at  r^  =  .70  through  .90). 

At  a  biserial  of  .30,  the  C50  test  coefficient  was  slightly  higher  and 
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at  rbi„  ■  .40  the  C50  conventional  and  .3  through  .7  1-PS  branching  tests 

had  the  largest  coefficients.  The  2-PS  branching  tests  compared  favorably 
with  the  best  conventional  test  only  at  very  high  biserials. 


Fifteen-Item  Tests 

Conventional  Tests.  All  fifteen-item  tests  were  evaluated  at  biserials 
of  .40,  .60,  and  .80.  The  test  score-ability  correlation  coefficients  for 
the  five  conventional  tests  are  given  in  the  first  five  rows  of  Table  4. 

The  C50  test  had  the  highest  coefficient  at  biserials  of  .40  and  .60.  At 
a  biserial  of  .80,  the  .1  through  .9  tests  (both  N  and  R)  did  best.  A 
comparison  of  the  .3  through  .7  and  .1  through  .9  tests  across  the  three 
biserials  showed  that  the  narrower  range  tests  received  higher  coeffi¬ 
cients  at  the  lower  biserials  (.40  and  .60)  and  the  wider  range  tests  did 
better  for  the  high  biserial  (.80).  This  general  trend  was  consistent 
with  the  results  obtained  for  the  five-  and  ten-item  conventional  tests. 

For  tests  of  a  given  range  of  item  difficulties,  those  with  approximately 
normally  distributed  item  difficulties  were  superior  to  those  with  recti¬ 
linear  difficulty  distributions  at  the  .40  bl serial  and  did  less  well  than 
their  rectilinear  counterparts  at  a  biserial  of  .80.  At  r^  ■  .60,  no 

difference  was  obtained  between  the  .3  through  .7  N  and  R  tests,  but  the 
.1  through  .9  N  test  was  superior  to  the  R  test  of  the  same  range.  This 
same  trend  was  also  found  for  the  ten- item  conventional  tests.  In 
general,  as  biserials  (and  thus  item  intercorrelations)  increased,  wider 
range  tests  and  tests  with  more  rectilineal  item  difficulty  distributions 
did  progressively  better. 

Branching  Tests.  Data  for  the  two  fifteen- item  branching  tests  are 
given  in  the  last  two  rows  of  Table  4.  The  .3  through  .7  teat  correlated 
higher  with  the  ability  criterion  at  the  .40  biaerial,  while  the  .1 
through  .9  test  yielded  the  highest  coefficient  at  r^g  ■  .80.  The  two 

branching  tests  were  essentially  equivalent  at  the  .60  biserial. 

Comparison  of  Conventional  and  Branching  Tests.  Both  branching  tests 
yielded  higher  coefficients  than  did  ny  of  the  conventional  tests  for 
biserials  of  .60  and  .80.  At  the  .40  biserial,  the  C50  test  was  essentially 
equivalent  to  the  .3  through  .7  branching  test. 


Effects  of  Test  Length 

Table  3  gives  the  increments  in  test  score-ability  coefficients  as 
the  teats  were  increased  in  length  from  five  to  fifteen  items.  Increasing 
the  number  of  items  from  five  to  ten  resulted  in  increments  in  correlation 
about  twice  as  large  as  those  obtained  by  increasing  test  length  from  ten 
to  fifteen  items.  Increases  in  test  length  led  to  higher  test  score-ability 
coefficients  for  the  lower  biserial  values.  There  appeared  to  be  little 
difference  between  conventional  and  branching  tests  in  terms  of  the  effects 
of  increasing  test  length. 
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Table  4 


TEST  SCORE-ABILITY  CORRELATION  COEFFICIENTS  FOR  FIFTEEN- ITEM- PER- SUBJECT 

CONVENTIONAL  AND  BRANCHING  TESTS 


Biberials 

.40 

.60 

.80 

Test 

Correlation  Coefficients* 

C  (all  .50) 

792 

896 

923 

C  (.5  -  -7,  N) 

787 

894 

928 

c  (.3  -  .7,  R) 

785 

894 

930 

C  (.1  -  .9,  N) 

764 

884 

936 

C  (.1  -  .9,  R) 

751 

877 

937 

B  (.3  -  -7,  1-PS) 

793 

903 

943 

B  (  .1  -  .9,  1-PS) 

786 

902 

953 

8 Decimal  points  omitted. 

Table 

5 

INCREMENTS  IN  TEST 
WITH 

SCORE-ABILITY  CORRELATION  COEFFICIENTS 
INCREASE  IN  TEST  LENGTH 

Biserials 

•  30 

.40 

.50 

.60  .70 

.80 

•90 

Correlation  Coefficients' 

ft 

C  (all  .50) 

5-10 

132 

127 

111 

090  068 

047 

027 

10-15 

064 

037 

018 

c  (.3  -  -7,  N) 

5-10 

10-15 

064 

038 

019 

C  (.1  -  .9,  N) 

5-10 

10-15 

062 

040 

023 

c  (.3  -  .7,  R) 

5-10 

131 

128 

113 

092  071 

050 

030 

10-15 

066 

040 

019 

C  (.1  -  .9,  R) 

5-10 

129 

131 

121 

104  084 

063 

041 

10-15 

071 

047  • 

024 

B  (  .3  -  .7,  1-PS) 

5-10 

134 

129 

114 

092  069 

046 

025 

10-15 

065 

037 

017 

B  (  .1  -  .9,  1-PS) 

5-10 

137 

133 

115 

094  071 

048 

025 

10-15 

066 

039 

018 

a  Decimal  points  omitted. 
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OVERVIEW 


Both  conventional  and  branching  test  data  showed  that  tests  with  the 
least  spread  of  item  difficul  !es  yielded  the  highest  correlation  coeffi¬ 
cients  with  underlying  ability  when  low  to  moderate  item  biserials  were 
assumed.  For  medium  to  high  biserials,  the  moderate  range  and  wide  range 
tests  tended  to  yield  coefficients  of  about  the  same  magnitude.  The  wide 
range  tests  generally  did  best  when  very  high  biserials  were  assumed. 

These  data  are  consistent  with  the  9“  and  lB-item  test  data  reported  by 
Brogden  (4).  The  shift  in  the  relative  effectiveness  of  the  narrower  and 
wider  range  tests  tended  to  take  place  earlier  when  test  length  was 
increased . 

For  the  lowest  biserial  assumed  ( .30),  the  C50  test  was  the  only  con¬ 
ventional  test  which  correlated  higher  with  the  ability  criterion  than  did 
the  best  branching  test.  At  biserials  of  .40  and  .50,  the  ten-  and 
fifteen-item  branching  tests  covering  a  .3  to  .7  range  and  the  C90  test 
were  essentially  equivalent.  For  biserials  of  .60  and  above,  one  of  the 
branching  tests  always  did  better  than  anv  of  the  conventional  tests.  In 
general,  the  differences  in  correlation  with  underlying  ability  were  small 
but  systematic.  Since  the  data  were  by  definition  errorless,  greater 
significance  may  be  attached  to  these  differences  than  would  be  the  case 
with  empirical  data. 

A  comparison  of  one- item- per- stage  and  two- item-per- stage  branching 
tests  (at  the  ten-item  test  length)  indicated  that  the  one-item-per-stage 
tests  had  uniformly  higher  coefficients  than  did  the  two-item-per-stage 
tests  of  the  same  range.  In  view  of  these  results,  it  would  not  seem 
profitable  to  use  the  more  complex  two-item-per-stage  structure  in  the 
development  of  branching  tests  for  the  purpose  of  maximizing  overall 
correlation. 
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11.  SOOMSOSIMS  MILITARY  ACTIVITY 

Office,  Chief  of  R'  search  and  Development, 
DA 

In  the  conduct  of  continuing  research  to  improve  screening  of  potential  enlisted 
manpower,  the  Enlisted  Manpower  Work  Unit  has  explored  the  potential  contribution  of 
tests  with  branching  programs.  Such  tests  permit  greater  individualization  of  testing 
than  do  conventional  tests  by  testing  with  items  appropriate  to  each  examinee's  ability 
level.  The  present  publication  reports  on  a  comparison  of  computer-simulated  branching 
and  conventional  tests  which  varied  in  length,  distribution  of  item  difficulty,  and 
item  validity.  Comparison  was  in  terms  of  correlation  between  underlying  ability  and 
test  scores  (Lord's  model).  The  model  assumes  that  there  is  a  trait  or  ability  under¬ 
lying  the  raw  scores  on  a  test  and  that  the  probability  of  examinee's  responding 
correctly  to  a  test  item  is  a  normal  ogive  function  of  his  position  on  the  ability 
dimension. 

The  principal  finding  was  that  in  tests  with  higher  item  validities  (r^  =  *60  - 

.  <0),  a  branching  test  had  higher  correlation  with  underlying  ability  than 
did  any  of  the  conventional  tests,  for  all  three  lengths  studied. 

This  theoretical  analysis  supported  an  exploratory  experimental  study  previously 
made  and  provides  useful  guidelines  for  further  research  with  branching  tests. 
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